Facial optical flow estimation via neural non-rigid registration

Abstract Optical flow estimation in human facial video, which provides 2D correspondences between adjacent frames, is a fundamental pre-processing step for many applications, like expression capture and recognition. However, it quite challenging as images contain large areas of similar textures, rich expressions, rotations. These characteristics also result the scarcity large, annotated real-world datasets. We propose robust accurate method to learn optical self-supervised manner. Specifically, we utilize various shape priors, including face depth, landmarks, parsing, guide learning task via differentiable nonrigid registration framework. Extensive experiments demonstrate that our achieves remarkable improvements presence significant expressions